Dual Attention on Pyramid Feature Maps for Image Captioning

Generating natural sentences from images is a fundamental learning task for visual-semantic understanding in multimedia. In this paper, we propose to apply dual attention on pyramid image feature maps fully explore the correlations and improve quality of generated sentences. Specifically, with full consideration contextual information provided by hidden state RNN controller, can better localize visually indicative semantically consistent regions images. On other hand, help re-calibrate importance components channel-wise dependencies, discriminative power visual features content description. We conducted comprehensive experiments three well-known datasets: Flickr8K, Flickr30 K MS COCO, which achieved impressive results generating descriptive smooth Using either convolution or more informative bottom-up features, composite model boost performance image-to-sentence translation, limited computational resource overhead. The proposed methods are highly modular, be inserted into various captioning modules further performance.